Customizing Packer With Plugins

Packer allows for building plugins that can be used as the following:

• A Packer builder

• A Packer provisioner

• A Packer post-processor

Builders are used when you need to interact with the system that will use your image: Docker, AWS, GCP, Azure, or others. As this isn't a common use outside cloud providers or companies such as VMware adding support, we will not cover this.

Post-processors are normally used to push an image to upload the artifacts generated earlier. As this isn't common, we will not cover this.

Provisioners are the most common, as they are part of the build process to output an image.

Packer has two ways of writing these plugins:

• Single-plugins

• Multi-plugins

Single plugins are an older style of writing plugins. The Goss provisioner is written in the older style, which is why we installed it manually.

With the newer style, packer init can be used to download the plugin. In addition, a plugin can register multiple builders, provisioners, or post-processors in a single plugin. This is the recommended way of writing a plugin.

Unfortunately, the official documentation for multi-plugins and doing releases that support packer init is incomplete at the time of this writing. Following the directions will not yield a plugin that can be released using their suggested process.

The instructions included here will fill in the gaps to allow building a multi-plugin that users can install using packer init.

Let's get into how we can write a custom plugin.

Writing your own plugin#

Provisioners are powerful extensions to the Packer application. They allow us to customize the application to do whatever we need.

We have already seen how a provisioner can execute Goss to validate our builds. This allowed us to make sure future builds follow a specification for the image.

To write a custom provisioner, we must implement the following interface:

The preceding code is described as follows:

• Line 2: ConfigSpec() returns an object that represents your provisioner's HCL2 spec. This will be used by Packer to translate a user's config to a structured object in Go.

• Line 3: Prepare() prepares your plugin to run and receives a slice of interface{} that represents the configuration. Generally, the configuration is passed as a single map[string]interface{}. Prepare() should do preparation operations such as pulling information from sources or validating the configuration, which should cause a failure before even attempting to run. This should have no side effects; that is, it should not change any state by creating files, instantiating VMs, or any other changes to the system.

• Line 4: Provision() does the bulk of the work. It receives a Ui object that is used to communicate with the user and Communicator that is used to communicate with the running machine. There is a provided map that holds values set by the builder. However, relying on values there can tie you to a builder type.

For our example provisioner, we are going to pack the Go environment and install it on the machine. While Linux distributions will often package the Go environment, they are often several releases behind. Earlier, we were able to do this by using file and shell (which can honestly do almost anything), but if you are an application provider and you want to make something repeatable for other Packer users across multiple platforms, a custom provisioner is the way to go.

Adding our provisioner configuration#

To allow the user to configure our plugin, we need to define a configuration. Here is the config option we want to support: Version (string)[optional], the specific version to download defaults to latest.

We will define this in a subpackage: internal/config/config.go.

In that file, we will add the following:

Unfortunately, we now need to be able to read this from an hcldec.ObjectSpec file. This is complicated, so HashiCorp has created a code generator to do this for us. To use this, you must install their packer-sdc tool:

To generate the file, we can execute the following from inside internal/config:

This will output a config.hcl2spec.go file that has the code we require. This uses the //go:generate line defined in the file.

Defining the plugin's configuration specification#

At the root of our plugin location, let's create a file called goenv.go.

So, let's start by defining the configuration the user will input:

This imports the following:

• Line 4: The config package we just defined.

• Lines 5–7: Three packages are required to build our plugin:

  • packer

  • plugin

  • version

• Line 8: A packerConfig package for dealing with HCL2 configs.

Note: The ... is a stand-in for standard library packages and a few others for brevity. We can see them all in the repository version.

Now, we need to define our provisioner:

This is going to hold our configuration, some file content, and the Go tarball filename. We will implement our Provisioner interface on this struct.

Now, it's time to add the required methods.

Defining the ConfigSpec() function#

ConfigSpec() is defined for internal use by Packer. We simply need to provide the spec so that Packer can read in the configuration.

Let's use config.hcl2spec.go we generated a second ago to implement ConfigSpec():

This returns ObjectSpec that handles reading in our HCL2 config.

Now that we have that out of the way, we need to prepare our plugin to be used.

Defining Prepare()#

Remember that Prepare() simply needs to interpret the intermediate representation of the HCL2 config and validate the entries. It should not change the state of anything.

Here's what that would look like:

This code does the following:

• Line 2: Creates our empty config.

• Lines 3–5: Decodes the raw config entries into our internal representation.

• Line 6: Puts defaults into our config if values weren't set.

• Line 7: Validates our config.

We could also use this time to connect to services or any other preparation items that are needed. The main thing is not to change any state.

With all the preparation out of the way, it's time for the big finale.

Defining Provision()#

Provision() is where all the magic happens. Let's divide this into some logical sections:

• Fetch our version

• Push a tarball to the image

• Unpack the tarball

• Test our Go tools installation

The following code wraps other methods that execute the logical sections in the same order:

This code calls all our stages (which we will define momentarily) and outputs some messages to the UI. The Ui interface is defined as follows:

Unfortunately, the UI is not well documented in the code or the documentation. Here is a breakdown:

• Line 2: You can use Ask() to ask a question of the user and get a response. As a general rule, you should avoid this, as it removes automation. Better to make them put it in the configuration.

• Lines 3–4: Say() and Message() both print a string to the screen.

• Line 5: Error() outputs an error message.

• Line 6: Machine() simply outputs a statement into the log generated on the machine using fmt.Printf() that is prepended by machine readable:.

• Line 7: getter.ProgressTracker() is used by Communicator to track download progress. You don't need to worry about it.

Now that we have covered the UI, let's cover Communicator:

Methods in the preceding code block are described as follows:

• Line 2: Start() runs a command on the image. You pass *RemoteCmd, which is similar to the Cmd type we used from os/exec in previous sections.

• Line 3: Upload() uploads a file to the machine image.

• Line 4: UploadDir() uploads a local directory recursively to the machine image.

• Line 5: Download() downloads a file from the machine image. This allows you to capture debugs logs, for example.

• Line 6: DownloadDir() downloads a directory recursively from the machine to a local destination. You can exclude files.

Let's look at building our first helper, p.fetch(). The following code determines what URL to use to download the Go tools. Our tool is targeted at Linux, but we support installing versions for multiple platforms. We use Go's runtime package to determine the architecture (386, ARM, or AMD 64) we are currently running on to determine which package to download. The users can specify a particular version or latest. In the case of latest, we query a URL provided by Google that returns the latest version of Go. We then use that to construct the URL for download:

This code makes the HTTP request for the Go tarball and then stores that in .content:

Now that we have fetched our Go tarball content, let's push it to the machine:

The preceding code uploads our content to the image. Upload() requires that we provide *os.FileInfo, but we don't have one because our file does not exist on disk. So, we use a trick where we write the content to a file in an in-memory filesystem and then retrieve *os.FileInfo. This prevents us from writing unnecessary files to disk.

Note: One of the odd things about Communicator.Upload() is that it takes a pointer to an interface (*os.FileInfo). This is almost always a mistake by an author. Don't do this in your code.

The next thing needed is to unpack this on the image:

This code does the following:

• Lines 2–3: Defines a command that unwraps our tarball and installs to /usr/local

• Lines 5–10: Wraps that command in *packerRemoteCmd and captures STDOUT and STDERR

• Lines 12–14: Runs the command with Communicator: If it fails, returns the error and STDOUT/STDERR for debug

The last step for Provisioner is to test that it installed:

This code does the following:

• Lines 4–10: Runs /usr/local/go/bin/go version to get the output.

• Line 11: If it fails, returns the error and STDOUT/STDERR for debug.

Now, the final part of the plugin to write is main():

This code does the following:

• Line 2: Defines our release version as "0.0.1".

• Line 3: Defines the release as a "dev" version, but you can use anything here. The production version should use "".

• Line 6: Initializes pv, which holds the plugin version information. This is done in init() simply because the package comments indicate it should be done this way instead of in main() to cause a panic at the earliest time if a problem exists.

• Line 13: Makes a new Packer plugin.Set:

  • Sets the version information. If not set, all GitHub releases will fail.

  • Registers our provisioner with the "goenv" plugin name:

    • Can be used to register other provisioners

    • Can be used to register a builder, set.RegisterBuilder(), and a post-processor, set.RegisterPostProcessor()

• Line 17: Runs Set we created and exits on any error.

We can register with a regular name, which would get appended to the name of the plugin. If using plugin.DEFAULT_NAME, our provisioner can be referred to simply by the plugin's name.

So, if our plugin is named packer-plugin-goenv, our plugin can be referred to as goenv. If we use something other than plugin.DEFAULT_NAME, such as example, our plugin would be referred to as goenv-example.

We now have a plugin, but to make it useful we must allow people to initialize it.

Note: In this exercise, we don't go into testing Packer plugins. As of the time of publishing, there is no documentation on testing. However, Packer's GoDoc page has public types that can mock various types in Packer to help test your plugin.

This includes mocking the Provisioner, Ui, and Communicator types to allow you to test. You can find these here.